A COMPREHENSIVE REVIEW ON MULTI-VIEW CHEST X-RAY 3D RECONSTRUCTION FOR ENHANCED THORACIC DISEASE DIAGNOSIS
Keywords:
Chest Radiography, X-ray Imaging, Medical Imaging, Neural Rendering, 3D Volume Reconstruction, 2D-to-3D Conversion, Deep Learning in Radiology, Computer Vision in Healthcare, Neural Implicit Representations, Volumetric Medical Data, AI in Diagnostic Imaging, Image-based Reconstruction, Clinical Imaging AnalysisAbstract
In thoracic medicine, accurate diagnosis is crucial to patient care. If not identified early and correctly, thax-related conditions such as pneumonia, tuberculosis, lung cancer, and other cardiopulmonary diseases can have serious health consequences. Therefore, one of the most important steps in improving patient outcomes is increasing diagnostic accuracy. Because of their affordability, speed, and accessibility, chest X-rays (CXRs) are one of the most widely used imaging methods; however, they are essentially 2D predictions of 3D anatomical structures. The depth information is compressed by this flat representation, which frequently results in overlapping anatomical features. Because of this, it may be difficult to determine the precise location, size, or depth of lesions, which could result in a misdiagnosis or misunderstanding. Techniques for multi-view 3D reconstruction are being investigated in order to get around the drawbacks of 2D imaging. The goal of these techniques is to use several 2D X-ray views to create a 3D volumetric representation of the thorax. Clinicians will gain a better understanding of spatial connections within the thoracic cavity as a result, which will be especially helpful for lung pathologies, tumor localization, and surgical planning. Recent developments in deep learning and 3D vision have given rise to Neural Radiance Fields (NeRFs) — deep neural networks that learn to encode 3D scenes from multiple 2D images. A variant specifically designed for the medical imaging setting, called MedNeRF (Medical Neural Radiance Fields), is utilized in this work. MedNeRF is made suitable for medical imaging settings and can learn the volumetric shape of the thorax from multiple X-ray views. NeRFs learn by optimizing a neural network to produce color and density values along rays intersecting a scene. For medicine, this enables the model to encode intricate internal structures such as ribs, lungs, and heart tissues into a coherent 3D model.
Downloads
References
J. Ni et al., "Enhance Chest X-ray Classification with Multi-image Fusion and Pseudo-3D Reconstruction," 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy, 2022, pp. 1-8, doi: 10.1109/IJCNN55064.2022.9892095.
Khan U, Yasin A, Abid M, Shafi I, Khan SA. A Methodological Review of 3D Reconstruction Techniques in Tomographic Imaging. J Med Syst. 2018 Sep 4;42(10):190. doi: 10.1007/s10916-018-1042-2. PMID: 30178184.
C. Koehler, T. Wischgoll and F. Golshani, "Reconstructing the human ribcage in 3d with x-rays and geometric models," in IEEE MultiMedia, vol. 17, no. 3, pp. 46-53, July-September 2010, doi: 10.1109/MMUL.2010.5692182.
Corona-Figueroa, A., Frawley, J., Bond-Taylor, S., Bethapudi, S., Shum, H. P., & Willcocks, C. G. (2022). MedNeRF: Medical Neural Radiance Fields for Reconstructing 3D-aware CT-Projections from a Single X-ray. . https://doi.org/10.1109/embc48229.2022.9871757
Yufeng Wang, Zhan-Li Sun, Zhigang Zeng, Kin-Man Lam, TRCT-GAN: CT reconstruction from biplane X-rays using transformer and generative adversarial networks, Digital Signal Processing, Volume 140, 2023, 104123, ISSN 1051-2004, https://doi.org/10.1016/j.dsp.2023.104123.
X. Ying, H. Guo, K. Ma, J. Wu, Z. Weng and Y. Zheng, "X2CT-GAN: Reconstructing CT From Biplanar X-Rays With Generative Adversarial Networks," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 10611-10620, doi: 10.1109/CVPR.2019.01087.
Shiode, R., Kabashima, M., Hiasa, Y. et al. 2D–3D reconstruction of distal forearm bone from actual X-ray images of the wrist using convolutional neural networks. Sci Rep 11, 15249 (2021). https://doi.org/10.1038/s41598-021-94634-2
Y. Gao et al., "3DSRNet: 3-D Spine Reconstruction Network Using 2-D Orthogonal X-Ray Images Based on Deep Learning," in IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1-14, 2023, Art no. 4506214, doi: 10.1109/TIM.2023.3296838.
Maken, P., Gupta, A. 2D-to-3D: A Review for Computational 3D Image Reconstruction from X-ray Images. Arch Computat Methods Eng 30, 85–114 (2023). https://doi.org/10.1007/s11831-022-09790-z
Hosseinian, S. and Arefi, H.: 3D RECONSTRUCTION FROM MULTI-VIEW MEDICAL X-RAY IMAGES – REVIEW AND EVALUATION OF EXISTING METHODS, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-1/W5, 319–326, https://doi.org/10.5194/isprsarchives-XL-1-W5-319-2015, 2015.
Dong, X., Xu, S., Liu, Y., Wang, A., Saripan, M. I., Li, L., Zhang, X., & Lu, L. (2020). Multi-view secondary input collaborative deep learning for lung nodule 3D segmentation. Cancer imaging : the official publication of the International Cancer Imaging Society, 20(1), 53. https://doi.org/10.1186/s40644-020-00331-0
Corona-Figueroa, A., Frawley, J., Bond-Taylor, S., Bethapudi, S., Shum, H., & Willcocks, C.G. (2022). MedNeRF: Medical Neural Radiance Fields for Reconstructing 3D-aware CT-Projections from a Single X-ray. 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 3843-3848.
Pechin Lo, Bram van Ginneken, Joseph M. Reinhardt, Tarunashree Yavarna, Pim A. de Jong, Benjamin Irving, Catalin Fetita, Margarete Ortner, Romulo Pinho, Jan Sijbers, Marco Feuerstein, Anna Fabijan- ska, Christian Bauer, Reinhard Beichel, Carlos S. Mendoza, Rafael Wiemker, Jaesung Lee, Anthony P. Reeves, Silvia Born, Oliver Weinheimer, Eva M. van Rikxoort, Juerg Tschirren, Ken Mori, Benjamin Odry, David P. Naidich, Ieneke Hartmann, Eric A. Hoffman, Mathias Prokop, Jesper H. Pedersen, and Marleen de Bruijne, “Extraction of airways from ct (exact’09),” IEEE Transactions on Medical Imaging, vol. 31, no. 11, pp. 2093–2107, 2012
Shoulie Xie, Weimin Huang, Tao Yang, Dajun Wu, and Huiying Liu, “Compressed sensing based image reconstruction with projection recovery for limited angle cone-beam ct imaging,” in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), 2020, pp. 1307–1310.
S. Delorme, "Assessment of the 3D Reconstruction and High-Resolution Geometrical Modeling of the Human Skeletal Trunk from 2D Radiographic Images", IEEE Trans. Biomedical Eng., vol. 50, no. 8, pp. 989-998, 2003.
S. Benameur, "Three-Dimensional Biplanar Reconstruction of Scoliotic Rib Cage Using the Estimation of a Mixture of Probabilistic Prior Models", IEEE Trans. Biomedical Eng., vol. 52, no. 10, pp. 1713-1728, 2005.
H. Lamecker, T. H. Wenckebach and H. Hege, "Atlas-Based 3D-Shape Reconstruction from X-Ray Images", Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 371-374, 2006.
H. Wechsler, "Image Processing Algorithms Applied to Rib Boundary Detection in Chest Radiographs", Computer Vision Graphics and Image Processing, vol. 7, no. 3, pp. 375-390, 1978.
F. Plourde, F. Cheriet and J. Dansereau, "Semi-Automatic Detection of Scoliotic Rib Borders Using Chest Radiographs", Studies in Health Technology & Informatics, vol. 123, pp. 533-537, 2006.
Tri Huynh, Yaozong Gao, Jiayin Kang, Li Wang, Pei Zhang, Jun Lian, and Dinggang Shen, “Estimating ct image from mri data using structured random forest and auto-context model,” IEEE Transactions on Medical Imaging, vol. 35, no. 1, pp. 174–183, 2016
Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, and Daniel Duckworth, “NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections,” in CVPR, 2021
Lee, S., Kim, H., Kim, H., Choi, J., and Cho, J., Comparison of image enlargement according to 3D reconstruction in a CT scan: Using an aneurysm phantom. J. Korean Phys. Soc. 72(7):805–810, 2018.
Lun, Z., Gadelha, M., Kalogerakis, E., Maji, S., and Wang, R.: 3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks. In: Proceedings of the International Conference on 3D Vision (3DV) 2017, 2017.
Li, G., Xie, H., Ning, H., Citrin, D., Capala, J., Maass-Moreno, R., Guion, P., Arora, B., Coleman, N., Camphausen, K., and Miller, R., Accuracy of 3D volumetric image registration based on CT, MR and PET/CT phantom experiments. J. Appl. Clin. Med. Phys. 9(4):17–36, 2008.
Ge Wang, Jong Chu Ye, Klaus Mueller, and Jeffrey A. Fessler, “Image reconstruction is a new frontier of machine learning,” IEEE Transactions on Medical Imaging, vol. 37, no. 6, pp. 1289–1296, 2018.
Yinsheng Li, Ke Li, Chengzhu Zhang, Juan Montoya, and GuangHong Chen, “Learning to reconstruct computed tomography images directly from sinogram data under a variety of data acquisition conditions,” IEEE Transactions on Medical Imaging, vol. 38, no. 10, pp. 2469–2481, 2019
Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger, “Graf: Generative radiance fields for 3d-aware image synthesis,” in Advances in Neural Information Processing Systems (NeurIPS), 2020.
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa, “pixelNeRF: Neural radiance fields from one or few images,” in CVPR, 2021
Ngoc-Trung Tran, Viet-Hung Tran, Ngoc-Bao Nguyen, Trung-Kien Nguyen, and Ngai-Man Cheung, “On data augmentation for gan training,” IEEE Transactions on Image Processing, vol. 30, pp. 1882–1897, 2021
Johnson AE, Pollard TJ, Berkowitz SJ, Greenbaum NR, Lungren MP, Deng CY, Mark RG, Horng S. MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific Data. 2019;6.
Johnson AE, Pollard TJ, Greenbaum NR, Lungren MP, Deng C-Y, Peng Y, Lu Z, Mark RG, Berkowitz SJ, Horng S. MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.07042. 2019.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” in Computer Vision –
ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and JanMichael Frahm, Eds., Cham, 2020, pp. 405–421, Springer International Publishing.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 https://creativecommons.org/licenses/by-nc-nd/4.0/
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
